Skip to content

Use PEP 757 API for fmpz <-> int conversions#358

Closed
skirpichev wants to merge 1 commit intoflintlib:mainfrom
skirpichev:pep757-conversion/159
Closed

Use PEP 757 API for fmpz <-> int conversions#358
skirpichev wants to merge 1 commit intoflintlib:mainfrom
skirpichev:pep757-conversion/159

Conversation

@skirpichev
Copy link

@skirpichev skirpichev commented Jan 23, 2026

Closes #159

This lacks implementation for Python < 3.14. Not sure how to do this better, conditional compilation seems to be deprecated by Cython.

To my surprise, #324 approach seems to be better. Here my benchmarks.

Export

Benchmark ref pr324 patch
1<<7 166 ns 169 ns: 1.02x slower 169 ns: 1.02x slower
1<<38 216 ns not significant 219 ns: 1.01x slower
1<<300 3.06 us 923 ns: 3.31x faster 787 ns: 3.88x faster
1<<3000 15.9 us 2.17 us: 7.34x faster 3.70 us: 4.29x faster
1<<10000 49.4 us 5.93 us: 8.33x faster 11.6 us: 4.27x faster
Geometric mean (ref) 2.88x faster 2.33x faster

Import

Benchmark ref pr324 patch
1<<7 454 ns 451 ns: 1.01x faster 459 ns: 1.01x slower
1<<38 471 ns 466 ns: 1.01x faster 475 ns: 1.01x slower
1<<300 4.80 us 1.91 us: 2.52x faster 1.10 us: 4.37x faster
1<<3000 23.6 us 3.07 us: 7.68x faster 4.17 us: 5.65x faster
1<<10000 73.0 us 6.78 us: 10.76x faster 12.5 us: 5.86x faster
Geometric mean (ref) 2.92x faster 2.69x faster
Details 
# bench-export.py

import os

import pyperf

_T = os.getenv('_T')
if _T == "gmpy2.mpz":
    from gmpy2 import mpz
elif _T == "gmp.mpz":
    from gmp import mpz
else:
    from flint import fmpz as mpz

cases = ['1<<7', '1<<38', '1<<300', '1<<3000', '1<<10000']
runner = pyperf.Runner()
for c in cases:
    i = eval(c)
    m = mpz(i)
    runner.bench_func(c, int, m)
# bench-import.py

import os

import pyperf

_T = os.getenv('_T')
if _T == "gmpy2.mpz":
    from gmpy2 import mpz
elif _T == "gmp.mpz":
    from gmp import mpz
else:
    from flint import fmpz as mpz

cases = ['1<<7', '1<<38', '1<<300', '1<<3000', '1<<10000']
runner = pyperf.Runner()
for c in cases:
    i = eval(c)
    runner.bench_func(c, mpz, i)

@oscarbenjamin
Copy link
Collaborator

oscarbenjamin commented Jan 23, 2026

conditional compilation seems to be deprecated by Cython.

The way to do it is to move the conditional compilation to C e.g.:

python-flint/src/flint/flintlib/types/fmpz_mod_mat_compat.pxd

Lines 4 to 14 in b2b71f4

cdef extern from *:
"""
/*
* fmpz_mod_mat function signatures were changed in FLINT 3.1.0
*/
#if __FLINT_RELEASE >= 30100 /* Flint 3.1.0 or later */
#define compat_fmpz_mod_mat_init(mat, rows, cols, ctx) fmpz_mod_mat_init(mat, rows, cols, ctx)
#define compat_fmpz_mod_mat_init_set(mat, src, ctx) fmpz_mod_mat_init_set(mat, src, ctx)
#define compat_fmpz_mod_mat_clear(mat, ctx) fmpz_mod_mat_clear(mat, ctx)
#define compat_fmpz_mod_mat_set(A, B, ctx) fmpz_mod_mat_set(A, B, ctx)

@oscarbenjamin
Copy link
Collaborator

oscarbenjamin commented Jan 23, 2026

To my surprise, #324 approach seems to be better.

Is there anything that can improve this either in CPython or GMP?

Or should we just conclude that for python-flint the approach in gh-324 is best?

The approach in gh-324 seems simpler and works for all supported CPython versions without needing any conditional compilation.

What are gmpy2 or python-gmp going to use?

@skirpichev
Copy link
Author

skirpichev commented Jan 24, 2026

The way to do it is to move the conditional compilation to C e.g.

Yes, makes sense. I think it should be possible to expose simple wrappers like gmpy2's mpz_set_PyLong(), which will have all needed fallbacks.

Is there anything that can improve this either in CPython or GMP?

I don't think that something can be impoved on CPython side, at least not for "big enough" integers. The PEP 757 API provides raw access to the "array of digits" view for CPython int's.

Though, maybe mpz_import/export() could be improved.

What are gmpy2 or python-gmp going to use?

Unfortunately, #324 approach can't be translated to C easily for all supported CPython versions. Probably, I'll keep the current solution, but will benchmark also new approach (PyLong_AsNativeBytes/PyLong_FromNativeBytes).

@oscarbenjamin
Copy link
Collaborator

oscarbenjamin commented Jan 24, 2026

maybe mpz_import/export() could be improved.

Without having looked at the code my guess would be that because these functions are quite general they end up being less efficient than PyLong_AsNativeBytes/PyLong_FromNativeBytes when dealing specificially with CPython's 30 bit limb format.

@oscarbenjamin
Copy link
Collaborator

oscarbenjamin commented Jan 24, 2026

Okay thanks @skirpichev for the benchmarks here. This PR was useful but I think what it proves is that we should go with gh-324 instead.

@oscarbenjamin oscarbenjamin mentioned this pull request Jan 24, 2026
Merged
@oscarbenjamin
Copy link
Collaborator

oscarbenjamin commented Jan 24, 2026

This PR was useful but I think what it proves is that we should go with gh-324 instead.

Actually perhaps it suggests that we should not use mpz_import/mpz_export but it might still be faster to use PEP 757 with some custom C code that is specialised for the 30 bit limb case.

@skirpichev skirpichev closed this Jan 24, 2026
@skirpichev skirpichev deleted the pep757-conversion/159 branch January 24, 2026 23:47
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

No reviews

Assignees

No one assigned

Labels

None yet

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

More efficient conversion from Python int to fmpz

2 participants

@skirpichev @oscarbenjamin